Use Github Action to deploy to AWS
Trong Minh Duc Hoang | Feb 7, 2023
Overview
In this blog, we will use this infrastructure to do a step-by-step demo.
![](img/system.drawio.png)
Prerequisite
- IAM credentials (AWS access key ID, AWS Secret access key, region name)
- Github repositories
- Docker
- NodeJs (yarn)
Deploy static website to S3 with Github Action
![](img/Untitled_Diagram.drawio_(1).png)
1. Create AWS S3 bucket and enable ACLs
![](img/Screenshot_2023-02-07_at_3.17.06_PM.png)
- Enable public access
![](img/Screenshot_2023-02-07_at_10.01.13_PM.png)
2. Create workflow for Github action
In this demo, we are deploying an ReactJs app. Let’s explain some steps to build this app:
- Run
yarn install
to install all dependencies.
- Run
yarn build
to build the app. The content of output folder will be pushed to S3. In my case, the output folder isdist
.
Create a workflow file inside the repository .github/workflows/deploy.yml
![](img/Screenshot_2023-02-07_at_3.38.21_PM.png)
name: Production Build
on:
pull_request:
push:
branches:
- main # the branch and action we want to trigger this workflow
jobs:
build:
runs-on: ubuntu-latest # We use Uubuntu to run this workflow on
strategy:
matrix:
node-version: [ 16.x ] # Set NodeJs version here
steps:
- uses: actions/checkout@v3 # Check out the repo
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3 # Install NodeJs in this Ubuntu
with:
node-version: ${{ matrix.node-version }}
- name: Yarn Install # Run command to install all dependencies
run: |
yarn install
- name: Production Build # Build the file, the output folder in this case is dist
run: |
yarn build
- name: Deploy to S3
uses: jakejarvis/s3-sync-action@master
with:
args: --acl public-read --delete
env: # All the keys we need from AWS will be passed into this action
AWS_S3_BUCKET: ${{ secrets.AWS_PRODUCTION_BUCKET_NAME }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ secrets.AWS_REGION }}
SOURCE_DIR: "dist" # The output folder of build step above (it depends on your app)
3. Set env secrets
As you can see, we use four secret keys in above workflow. We need to define it in our repository’s setting and give it a value.
![](img/Screenshot_2023-02-07_at_3.44.48_PM.png)
4. Push code to Github and enjoy the automation
Github Action will automatically trigger this workflow when we push this workflow definition along with the code to the repo
To learn more about Github Action, you could go to this link to find an action that fits your need https://github.com/marketplace?type=actions&query=s3+
5. Access S3 from Cloudfront
Create a new distribution on Cloudfron to serve files from the S3 bucket. Now we can access our app publicly through Cloudfront.
6. Do some tricks after deployed
- There is a problem each time we deploy the app, which is cached files on Cloudfront, so we won’t see new changes on the website because the cached files are valid within 24 hours. That is why we want to create invalidation after each deployment. The below code shows a function to do it.
- We also want to know when is the deployment finished. In this case, we will use SNS to send an email to our inbox to inform the deployment is finished and ready for production.
a. Create a role for Lambda
Create a role for this Lambda that are enabled to publish message to SNS and create Cloudfront invalidation
![](img/Screenshot_2023-02-07_at_10.17.40_PM.png)
b. Create a SNS topic and subcribe our email to it.
c. Create a Lambda
Create a Lambda NodeJs 18 and assign above role to it.
Fill out YOUR_REGION
, YOUR_TOPIC_ARN
, and YOUR_DISTRIBUTION_ID
.
import { SNSClient, PublishCommand } from "@aws-sdk/client-sns";
import { CloudFrontClient, CreateInvalidationCommand } from "@aws-sdk/client-cloudfront";
const YOUR_REGION = 'YOUR_REGION';
const YOUR_TOPIC_ARN = 'YOUR_TOPIC_ARN';
const YOUR_DISTRIBUTION_ID = 'YOUR_DISTRIBUTION_ID';
const sendNotiToEmail = async () => {
const client = new SNSClient({ region: YOUR_REGION });
const params = {
Message: `New version deployed at ${new Date(Date.now()).toLocaleString("en-US", { timeZone: "America/Chicago" })}`,
TopicArn: YOUR_TOPIC_ARN
};
const command = new PublishCommand(params);
const response = await client.send(command);
return response;
}
const clearCloudFrontCache = async () => {
const client = new CloudFrontClient({ region: YOUR_REGION });
const command = new CreateInvalidationCommand({
DistributionId: YOUR_DISTRIBUTION_ID,
InvalidationBatch: {
CallerReference: new Date().getTime().toString(),
Paths: {
Quantity: 1,
Items: ['/*'] // Clear all cached files
}
}
});
return client.send(command);
}
export const handler = async (event) => {
await sendNotiToEmail();
await clearCloudFrontCache();
return {
statusCode: 200,
body: "Done"
}
};
d. Create a trigger from S3 bucket
Create a trigger from S3 bucket when index.html
has an update to run this lambda function after deployed
![](img/Screenshot_2023-02-07_at_10.28.37_PM.png)
Deploy backend NextJs application to ECS
![](img/Untitled_Diagram.drawio_(2).png)
1. Create security groups
a. Create a security group for load balancer
Allow all access from the internet (MyLoadBalancerSg
)
![](img/Screenshot_2023-02-07_at_10.55.25_PM.png)
b. Create a security group for ECS container
Only allow access from MyLoadBalancerSg
![](img/Screenshot_2023-02-08_at_1.02.43_PM.png)
2. Create a repository in ECR
![](img/Screenshot_2023-02-07_at_11.00.26_PM.png)
3. Build & push first image
We will dockerize our NextJs application and push it to ECR repository.
- Create a file
Dockerfile
inside our code base
# Use an official Node.js image as the base image
FROM node:16-alpine
# Set the working directory in the image to /app
WORKDIR /app
# Copy the rest of the application code to the image
COPY . .
# Install the application dependencies
RUN yarn install
# Build nextjs application, output folder is .next
RUN yarn build
# Specify the command to start the Next.js application
CMD ["npm", "run", "start"]
EXPOSE 3000
- In our ECR repository, click on
View push commands
then follow these steps to push our first image to this repo
![](img/Screenshot_2023-02-08_at_12.56.30_PM.png)
4. Create task definition
- Go to ECS, click on Task Definition, and click on Create Task Definition
- Create a task definition using the URI from above ECR (e.g:
http://856210122328.drecr.us-east-1.amazonaws.com/mywebiste-be:latest
)
- Map the port we want to expose from containers. In this case, NextJs’ port is 3000
- Pass env variables as your wish to containers
![](img/Screenshot_2023-02-07_at_11.10.42_PM.png)
5. Create ECS cluster and service using Cloudformation
We will use this Cloudformation template to create the rest of the service. We can still do it manually on the UI but when we create an ECS service inside the ECS cluster, there is a glitch that makes the Load Balancer section disappear so we cannot create a load balancer along with our ECS service. By doing it, Cloudformation will make sure we always have a load balancer when creating a ECS service.
- Create a yaml file for this template (e.g:
myecsstack.yml
)
- Replace all the parameters in the template
AWSTemplateFormatVersion: 2010-09-09
Description: The template used to create an ECS Service from the ECS Console.
Parameters:
# (REPLACE THIS) Enter the security group IDs for the load balancer and the container that were created in the previous step.
LoadBalancerSecurityGroup:
Type: CommaDelimitedList
Default: sg-04e9e5c992ae5d43c
ContainerSecurityGroup:
Type: CommaDelimitedList
Default: sg-011d187169c541958
# (REPLACE THIS) Enter the subnet IDs for the VPC where the load balancer and the container will be created.
SubnetIDs:
Type: CommaDelimitedList
Default: >-
subnet-0f1cffba22f6b4742,subnet-01a6bb4ca617d76e6,subnet-066dd3cb1343aa92d,subnet-06431437c9d252d83,subnet-05ece4ddfab51758e,subnet-0d719136bbfff4870
VpcID:
Type: String
Default: vpc-04b4663f76ce16dd8
# (REPLACE THIS) Create a new role with `AmazonECSTaskExecutionRolePolicy` and enter its arn here
TaskRole:
Type: String
Default: arn:aws:iam::856210122328:role/ecsTaskExecutionRole
# (REPLACE THIS) Enter the task definition's arn that we created from above step
TaskDefinitionArn:
Type: String
Default: arn:aws:ecs:us-east-1:856210122328:task-definition/mywebsite:1
# Choose whatever you want for below parameters
ECSClusterName:
Type: String
Default: mywebsite-cluster
LoadBalancerName:
Type: String
Default: MyLoadBalancerECS
ServiceName:
Type: String
Default: mywebsite-service
ContainerName:
Type: String
Default: mywebsite
ContainerPort:
Type: Number
Default: 3000 # matching with the exposed port from docker
TargetGroupName:
Type: String
Default: MyTargetGroupECS
Resources:
ECSCluster:
Type: 'AWS::ECS::Cluster'
Properties:
ClusterName: !Ref ECSClusterName
ECSService:
Type: 'AWS::ECS::Service'
Properties:
Cluster: !Ref ECSClusterName
CapacityProviderStrategy:
- CapacityProvider: FARGATE
Base: 0
Weight: 1
TaskDefinition: !Ref TaskDefinitionArn
ServiceName: !Ref ServiceName
SchedulingStrategy: REPLICA
DesiredCount: 1
LoadBalancers:
- ContainerName: !Ref ContainerName
ContainerPort: !Ref ContainerPort
LoadBalancerName: !Ref 'AWS::NoValue'
TargetGroupArn: !Ref TargetGroup
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: ENABLED
SecurityGroups: !Ref ContainerSecurityGroup
Subnets: !Ref SubnetIDs
PlatformVersion: LATEST
DeploymentConfiguration:
MaximumPercent: 200
MinimumHealthyPercent: 100
DeploymentCircuitBreaker:
Enable: true
Rollback: true
DeploymentController:
Type: ECS
ServiceConnectConfiguration:
Enabled: false
Tags:
- Key: 'ecs:service:stackId'
Value: !Ref 'AWS::StackId'
EnableECSManagedTags: true
DependsOn:
- Listener
LoadBalancer:
Type: 'AWS::ElasticLoadBalancingV2::LoadBalancer'
Properties:
Type: application
Name: !Ref LoadBalancerName
SecurityGroups: !Ref LoadBalancerSecurityGroup
Subnets: !Ref SubnetIDs
TargetGroup:
Type: 'AWS::ElasticLoadBalancingV2::TargetGroup'
Properties:
HealthCheckPath: /
Name: !Ref TargetGroupName
Port: !Ref ContainerPort
Protocol: HTTP
TargetType: ip
HealthCheckProtocol: HTTP
VpcId: !Ref VpcID
Listener:
Type: 'AWS::ElasticLoadBalancingV2::Listener'
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref TargetGroup
LoadBalancerArn: !Ref LoadBalancer
Port: 80
Protocol: HTTP
- After done with the template, go to Cloudformation and upload this file to create these services.
- Check if the load balancer is working as expected
6. Create workflow for Github Action
There is an official workflow to deploy AWS ECS on Github. You could go to your repository Click on Actions
\ New workflow
, and search for ecs
![](img/Screenshot_2023-02-08_at_1.25.16_PM.png)
But, in this guideline, we will customize the official workflow a little bit.
- We will download task definition and export it to a json file instead of providing a preset task definition json in our repo.
- Add a last step to send email to our inbox after everything is done
Notify SNS
using the same SNS topic that we created from previous section so we know when everything is done.
name: Deploy to Amazon ECS
on:
push:
branches: [ "main" ]
env:
AWS_REGION: ${{ secrets.AWS_REGION }} # set this to your preferred AWS region, e.g. us-west-1
ECR_REPOSITORY: ${{ secrets.ECR_REPOSITORY }} # set this to your Amazon ECR repository name
ECS_SERVICE: ${{secrets.ECS_SERVICE}} # set this to your Amazon ECS service name
ECS_CLUSTER: ${{secrets.ECS_CLUSTER}} # set this to your Amazon ECS cluster name
CONTAINER_NAME: ${{secrets.CONTAINER_NAME}} # set this to the name of the container in the
ECS_TASK_DEFINITION_NAME: ${{secrets.ECS_TASK_DEFINITION_NAME}}
permissions:
contents: read
jobs:
deploy:
name: Deploy
runs-on: ubuntu-latest
environment: production
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Download task definition
run: |
aws ecs describe-task-definition --task-definition "${{secrets.ECS_TASK_DEFINITION_NAME}}" --query taskDefinition > task-definition.json
- name: Build, tag, and push image to Amazon ECR
id: build-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
IMAGE_TAG: ${{ github.sha }}
run: |
# Build a docker container and
# push it to ECR so that it can
# be deployed to ECS.
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
- name: Fill in the new image ID in the Amazon ECS task definition
id: task-def
uses: aws-actions/amazon-ecs-render-task-definition@v1
with:
task-definition: task-definition.json
container-name: ${{ env.CONTAINER_NAME }}
image: ${{ steps.build-image.outputs.image }}
- name: Deploy Amazon ECS task definition
uses: aws-actions/amazon-ecs-deploy-task-definition@v1
with:
task-definition: ${{ steps.task-def.outputs.task-definition }}
service: ${{ env.ECS_SERVICE }}
cluster: ${{ env.ECS_CLUSTER }}
wait-for-service-stability: true
- name: Notify SNS
run: |
aws sns publish --topic-arn "${{secrets.SNS_ARN}}" --message "New Backend Deployment at $(date +%c)"
a. Set env variables that we need for this workflow
- AWS_REGION
- ECR_REPOSITORY
- ECS_SERVICE
- ECS_CLUSTER
- CONTAINER_NAME
- ECS_TASK_DEFINITION_NAME
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- SNS_ARN
![](img/Screenshot_2023-02-08_at_1.38.14_PM.png)
b. Create the workflow file in the source code and push it to Github
![](img/Screenshot_2023-02-08_at_1.46.15_PM.png)
![](img/Screenshot_2023-02-08_at_1.38.55_PM.png)
Finally, a new task definition will be deployed in the ECS service and replace the previous version. Its private IP will be automatically registered to the target group that we created along with the load balancer and ECS service.
There is something you can do to optimize this workflow:
- Reduce the size of the docker image
- Clean up the unused images in ECR because it will cost us a lot